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(57) Abstract 

A method and system are provided for effecting interactive, three-dimensional renderings 
of selected body organs for purposes of medical obsenration and diagnosis. A series of CT 
images of the selected body organs ait acquired. The series of CT images is stacked to fcmn 
a three-dimensional volume file. To facilitate interactive three-dimensional renderings the thiee- 
dinensional volume file may be subjected to an optional dataset reduction procedure to reduce 
pixel resolution and/or to divide the three-dimensional volume file into selected subvolumes. From 
a selected volume or subvolume. the image of a selected body organ is segmented of isolated. 
A wireframe model of the segmented organ image is then generated to enable interactive, three- 
dimensional rendering of the selected organ. 



40 



"n|^ mnocrwcmmnow 



CXTMCnON 



**'S^raAMafiii 



I 



•0 



I 



VOUMCPQMUCnON 



MDUCriOM 



mMCNTATlON 



FOR THE PURPOSES OF iNFORMATiON ONLY 



Codes used to identify States paity to tbe PCT on the front pages of pamphlets publishing international 
applications under the PCT. 



AT 


Aomia 


GB 


UnindKinsdoa 


MR 


Mmiunia 


AU 


Autfilia 


GE 


Oeocfia 


MW 


Malawi 


BB 


Bob^ 


GN 


Guinea 


NE 


Niger 


BE 


Bdgiam 


GR 


Graeoe 


NL 


Nettefln^ 


BF 


Burkiu Fiio 


HU 




NO 


Norway 


BG 


Butpria 


IE 


Ireland 


NZ 


New Zealand 


BJ 


n 


Italy 


PL 


Poland 


BR 


Bmil 


JP 


Japan 


FT 


Ponugil 


BY 


Beta 


KE 


Kenya 


RO 


Ronunta 


CA 


Ob^ 


KG 


Kyi (yuan 


RU 


Ruaiian Fcdention 


CF 


Cooil Africaa Republic 


KP 


Democnck Feopk't Republic 


SD 


Sudn 


CG 


CooBO 




of Xoiea 


SE 


Sweden 


CH 


SwneriMid 


m 


Republic of Kofca 


SI 


Skrvcnia 


a 


Cte d'lvoiit 


KZ 


KaukintaD 


SK 


Skmkia 


CM 




U 




SN 


SctKial 


CN 


China 


LK 


Sri Lanka 


TO 


Chad 


CS 


Cxectetlowakia 


LU 




TO 


Togo 


CZ 


C»cb RqniMac 


LV 


Ltfvia 


TJ 


Tajikiittn 


DB 


Gcfnuoy 


MC 


Monaco 


TT 


Trinidad and Tobago 


DK 




MD 


Repubbe of Moldova 


UA 


Uknine 


ES 


Spain 


MG 


MadacaKar 


US 


Uoiied Stales of America 


F1 


Fialad 


ML 


Mali 


U£ 


UcbekiiUD 


FR 


Pnooc 


MN 


MoDfoUa 


VN 


Viei Nun 


CA 


GaboD 











wo 96/13207 



PCT/US95/14022 



MSTHOD AND SYSTEM FOR PRODUCING INTERACTIVE, 
THREE-DIMENSIONAL RENDERINGS OF SELECTED BODY 
ORGANS RAVING HOLLOW LUMENS TO ENABLE SIMULATED 
MOVEMENT THROUGH THE LUMEN 



BacXaround of the Invention 

For many forms of cancer, early detection is 
essential for a favorable prognosis. The cancerous 
growth must be detected at an early stage before the 
10 cancer is allowed to grow and spread. Such is the 
case for colorectal and lung cancers* As a result, 
techniques have been developed to examine the colon 
and tracheobronchial airways for the growth of 
precancerous and cancerous masses. 

15 Colon cancer is the second leading cause of 

cancer death in the United States today. Fortunately, 
most colorectal carcinomas arise from preexisting 
adenomatous polyps, and the risk of cancer is directly 
related to the size of the polyp (1% in polyps less 

20 than 1 cm, 10% in polyps between 1 and 2 cm, and 30% 
in polyps 2 cm or greater) . Scientific studies 
suggest that the early detection and removal of small 
carcinomas and precursor adenomatous polyps reduces 
mortality. Therefore, current strategies for colon 

25 cancer screening focus on the early discovery of 

polyps. The techniques used to screen for colorectal 
cancer include flexible sigmoidoscopy (the use of a 
fiberoptic scope to examine the distal half of the 
colon) and fecal occult blood testing (wherein 

30 hemorrhage is detected) • There is some debate on the 
effectiveness of colorectal cancer screening, but it 
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has been predicted that a 30-40% reduction in 
mortality can be achieved with proper screening using 
a combination of fecal occult blood testing and 
sigmoidoscopy. 

5 The National Cancer Institute and the 

American Cancer Society recommend colorectal cancer 
screening for persons of average risk who are more 
than 50 years old using sigmoidoscopy and annual fecal 
occult blood tests. The fecal occult blood test is 

10 easy to perform but is plagued by many false positive 
and false negative results. Sigmoidoscopy suffers 
from the fact that it only examines the distal half of 
the colon (the rectum and the sigmoid colon) . This is 
a serious shortcoming since approximately 40% of 

15 colorectal carcinomas occur proximal to the splenic 
flexure and are therefore undetectable using 
sigmoidoscopy. 

Examination of the entire colon by a barium 
enema or conventional colonoscopy increases 

20 sensitivity for cancer detection but also increases 
the risks and costs. A barium enema causes patient 
discomfort and/ or embarrassment and exposes the 
patient to substantial radiation. Colonoscopy, like 
sigmoidoscopy! does not always examine the entire 

25 colon since the cecum is not reached in approximately 
15% of colonoscopies. In addition, colonoscopy 
requires patient sedation, places the patient at risk 
for bowel perforation, and is comparatively expensive. 
Furthermore, with the exception of fecal occult blood 

30 testing, all of these procedures meet with significant 
patient discomfort. 
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Turning now to the tracheobronchial 
examination, Transbronchial Needle Aspiration (TBNA) 
is a bronchoscopy technique that permits the 
outpatient diagnosis and staging of mediastinal 
5 disease. This procedure allows for the outpatient 
sampling of tissue specimens that might otherwise 
require a surgical procedure. With TBNA, a needle is 
placed through an airway wall in the vicinity of a 
suspected lesion to retrieve a tissue sample. 

10 Conventionally, the bronchoscopist is guided only by a 
mental model of the patient's anatomy and pathology 
following review of bronchoscopy images and/ or a 
series of thoracic computed tomography (CT) images. 
As can be expected, proper placement of the needle can 

15 be extremely difficult and to a small degree somewhat 
imprecise. 

Accordingly, it is highly desirable to have 
a reliable, efficient method for examining the 
tracheobronchial tree and/ or the colon of a patient to 

2 0 detect early cancer. The technique should allow for 
the discovery of polyps of 1 cm or greater in size in 
the colon and 5 mm or greater in the airways. 
Preferably, the method should reduce the amount of 
discomfort encountered by the patient, decrease the 

25 risk of injury to the patient, and be conducted in a 
reasonable amount of time without being prohibitively 
expensive. Preferably, the method should be non- 
invasive or minimally invasive. 

Stunmarv of the Invention 

30 In accordance with the present invention, a 

system and method are provided for producing two- 
dimensional images of a selected structure, such as a 
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portion of the body, to enable the creation of a 
three-dimensional rendering of the selected structure. 
More specifically, two-dimensional images of a 
selected body portion are acquired with use of a 
scanner, for example, a helical computed tomography 
(CT) scanner. The two-dimensional images are then 
stacked to create a three-dimensional image volume. 
From the three-dimensional volume, image features of 
one or more selected body organs are isolated or 
segmented. Isosur faces of the segmented organs are 
produced and wireframe models are then generated from 
each of the isosurfaces for the respective segmented 
organs. The wireframe models are used to generate 
real time, three-dimensional images (renderings) of 
the selected organs. 

In a specific application for generating a 
three-dimensional rendering of a patient's colon, the 
patient initially undergoes a selected preparation 
procedure. For example, the patient's colon is 
20 initially cleansed and then inflated with air to 

permit the acquisition of unobstructed two-dimensional 
images of the colon. Next, the patient undergoes a CT 
scan to produce a series of two-dimensional images of 
the patient's internal organs. Preferably, a spiral 
25 or helical CT scanner is employed to provide a series 
of uninterrupted two-dimensional images through the 
body. The series of two-dimensional images are 
transferred from the scanner to a graphics computer to 
effect various image processing procedures. The 
30 dataset corresponding to the series of two-dimensional 
images may be transferred to the graphics computer in 
a compressed format for decompression on the graphics 
computer. Alternatively, the dataset representing the 
series of two-dimensional images may be decompressed 
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on the computer console of the scanner prior to 
transfer to the graphics computer. 

After transfer to the graphics computer, the 
series of two-dimensional images are stacked in order 
5 to form a three-dimensional volume file. In order to 
facilitate three-dimensional rendering of a selected 
organ contained within the three-dimensional volume of 
images, the three-dimensional volume file may be 
subjected to various optional dataset reduction 
10 techniques. For example, a reduction of pixel 

resolution on the series of two-dimensional images may 
be effected. In addition, the three-dimensional 
volume file may be separated into selected subvolumes. 

After the optional dataset reduction 

15 procedure is completed, an image segmentation process 
is performed in order to isolate features of a 
selected organ or region of interest from the three- 
dimensional volume file. Image segmentation may be 
effected by various techniques. For example, an image 

20 slice through the three-dimensional volume file may be 
subjected to a thresholding process in which a 
physical property of the two-dimensional image slice, 
such as x-ray attenuation, may be used to establish a 
particular threshold range, such as a range of x-ray 

25 attenuation values, that corresponds to the organ of 
interest. After an appropriate threshold range is 
determined, the entire three-dimensional volume file 
is then thresholded to segment the organ of interest. 
For example, in order to segment the colon, a 

30 threshold range corresponding to the air column within 
the colon could be selected to isolate the inner wall 
of the colon. 
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An alternative segmentation technique may be 
employed in which a region growing technique is used 
to isolate the air column within the colon. Using the 
region growing technique, a "seed" is planted by 
5 selecting a data point or voxel within the air column 
of the colon. Neighboring voxels are progressively 
tested for compliance with a selected acceptance 
criteria^ such as x-ray attenuation values falling 
within a selected threshold range representing air. 
10 As such, the seed region continues to expand or grow 
until the entire air column within the lumen of the 
colon is filled. 

A surface, or isosurface, of the air column 
representing the colon is then produced. A wireframe 

15 model of the isosurface is then generated using a 

selected image processing technique such as a marching 
cubes algorithm. From the wireframe model of the 
colon, a three-dimensional interactive rendering is 
produced that enables the user to rapidly view a 

20 series of three-dimensional images of the lumen of the 
colon for purpose of detection of pathological 
conditions. As such, a user has the perception of 
flying through the lumen and virtual reality 
environment is achieved. 

25 Other procedures may be employed with the 

interactive, three-dimensional rendering of the organ 
for purposes of medical diagnosis or observation. For 
example, a three-dimensional rendering of the colon 
may be split open so that the interior half of the 

30 colon along a selected length may be viewed. In 
addition, selected flights or paths of movement 
through the lumen of the colon may be recorded on 
videotape or, alternatively, stored in the memory of 
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the graphics computer as a path of coordinates for 
subsequent reproduction and display. In addition, 
various areas of pathology may be marked by different 
colors during a flight through the lumen of the colon 
5 to facilitate subsequent detection. 

The method and system of the present 
invention may also be used to isolate other organs. 
For example, the system and method may be used to 
isolate the tracheobronchial airways, as well as the 
10 surrounding lymph nodes and blood vessels. During an 
interactive, three-dimensional rendering, a selected 
organ may be rendered transparent to facilitate or 
permit a view of the remaining 

(surrounding/neighboring) organs. For example, the 
15 airway walls may be made transparent so that the 
surrounding blood vessels and lymph nodes become 
visible from a perspective within the interior of the 
airways. 

Brief Description of the Dravinas 

20 The foregoing summary, as well as the 

following detailed description of the preferred 
embodiments of the present invention, will be better 
understood when read in conjunction with the 
accompanying drawings, in which: 

25 Fig. 1 is a flowchart representing a method 

in accordance with the present invention of producing 
interactive, three-dimensional renderings of selected 
structures such as a selected body organ; 

Fig. 2 is a flowchart representing a general 
3 0 process in accordance with the present invention of 
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converting two-dimensional images of a selected body 
organ into a three-dimensional image; 

Fig. 3 is a block diagram of a system used 
in the method of the present invention; 

5 Fig. 4 is a flowchart representing the steps 

involved in a data reduction process used in the 
method of the present invention; 

Fig. 5 is a flowchart representing the steps 
involved in segmentation of an image of the selected 
10 body organ from a three-dimensional volume; 

Fig. 6 is a flowchart representing the steps 
involved in a region growing procedure for 
segmentation of an image of the selected body organ 
from a three-dimensional volume; 

15 Fig. 7 is a perspective view of a three- 

dimensional rendering showing the selection of 
coordinates X„i„, X^, Y^, Y^, Z^, and to 
define a selected subvolume containing a colon; 

Fig. 8 is a view of a thresholded two- 
20 dimensional image slice of a colon displayed together 
with the corresponding gray-scale image; 

Fig. 9 is a representation of a wireframe 
model of a selected portion of the colon; 

Fig. 10 is a three-dimensional rendering of 
25 the portion of the colon represented by the wireframe 
model shown in Fig. 9; 
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Fig. 11 is an external perspective view of a 
three-dimensional rendering of a selected portion of 
the colon; 

Fig. 12 is a split-open view of the selected 
5 portion of the colon shown in Fig. 11; 

Fig. 13 is a map view showing a three- 
dimensional rendering of the colon on which a location 
marker is superimposed on the image; 

Fig. 14 is a three-dimensional rendering of 
10 a selected section of the colon corresponding to the 
location of the marker shown in Fig. 13; 

Fig. 15 is a perspective view of a three- 
dimensional rendering showing the selection of 
coordinates X^, X^,, Y^, Y^,, 2^, and 2^ to 
IS define a selected subvolume containing a 
tracheobronchial tree; 

Fig. 16 is a view of a thresholded two- 
dimensional image slice of a tracheobronchial tree 
displayed together with the corresponding gray-scale 
20 image; 

Fig. 17 is a representation of a wireframe 
model of a selected portion of the tracheobronchial 
tree; 

Fig. 18 is a three-dimensional rendering of 
25 the portion of the tracheobronchial tree represented 
by the wireframe model shown in Fig. 17; 
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Fig. 19 is an external perspective view of a 
three-dimensional rendering of a selected portion of 
the tracheobronchial tree; 

Fig. 20 is a split-open view of the selected 
5 portion of the tracheobronchial tree shown in Fig. 19; 

Fig. 21 is a map view showing a three- 
dimensional rendering of the tracheobronchial tree on 
which a location marker is superimposed on the image; 

Fig. 22 is a three-dimensional rendering of 
10 a selected section of the tracheobronchial tree 

corresponding to the location of the marker shown in 
Fig. 21; 

Fig. 23 is a diagrammatic view depicting a 
model for determining a center line through a lumen; 

15 Fig. 24 is a flowchart representing a 

procedure for making a three-dimensional rendering of 
a selected organ transparent relative to three- 
dimensional renderings of remaining organs; 

Fig. 25 is a flowchart representing a 
20 procedure for establishing "go to" points at selected 
locations on a three-dimensional rendering to cause 
the display of a three-dimensional rendering of the 
specific location selected by a respective "go to" 
point; 

25 Fig. 26 is a diagram depicting a pick point 

procedure in which a selected pick point on a three- 
dimensional image causes the display of three 
orthogonal planes passing through the pick point; and 
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Fig. 27 is a flowchart showing the steps 
involved in morphological dilation. 

Detailed Description of the Preferred Embodiments 

The present invention generally relates to a 
5 method and system, as schematically represented in 
Figs. 1, 2, and 3, for generating and displaying 
interactive, three-dimensional structures. The three- 
dimensional structures are in the general form of 
selected regions of the body and, in particular, body 

10 organs with hollow lumens such as colons, 

tracheobronchial airways, blood vessels, and the like. 
In accordance with the method and system of the 
present invention, interactive, three-dimensional 
renderings of a selected body organ are generated from 

15 a series of acquired two-dimensional images. 

As illustrated in Fig. 3, a scanner 22, such 
as a spiral or helical CT (Computed Tomography) 
scanner, operated by a computer console 24 is used to 
scan a selected three-dimensional structure, such as a 

2 0 selected anatomy, thereby generating a series of two- 
dimensional images 12 through that structure. A 
general procedure for converting or transforming the 
set of two-dimensional images 12 of the structure into 
a three-dimensional rendered image 17 is schematically 

25 illustrated in Fig. 2. Any type of digital image (CT, 
MR, US, SPECT, PET) is amendable to three-dimensional 
image processing. However, the acquisition of 
contiguous thin slices is imperative for generating 
acceptable three-dimensional images. Recent 

30 developments of volume acquisition devices such as 

spiral/helical CT, three-dimensional MRI, and three- 
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dimensional ultrasound equipment make it feasible to 
render three-dimensional images of any organ system. 

Referring to Fig. 2, each of the individual 
two-dimensional images 12 defines a two-dimensional 
5 (X-Y) matrix of picture elements, or pixels, with each 
pixel representing a predetermined physical property 
associated with the three-dimensional structure or 
organ at a particular location within the three- 
dimensional structure. Successive two-dimensional 

10 images 12 are spaced in a third-dimensional direction 
(Z) throughout the three-dimensional structure. The 
two-dimensional images 12 are typically obtained from 
a helical computed tomography (CT) scanner 22 operated 
by a computer console 24. For example, the scanner 

15 may be a General Electric HiSpeed Advantage Helical CT 
Scanner connected with an optional General Electric 
Independent computer console or physician's console. 
The computer console 24 may, however, be an integral 
part of the helical CT scanner 22 instead of a 

20 separate independent console. The two-dimensional 
images 12 can also be obtained from ultrasound, 
positron emission tomography, emission computed 
tomography, and magnetic resonance imaging. The 
physical property measured is directly associated with 

25 the scanning technique used to produce the two- 
dimensional images 12. For CT images the physical 
property measured is x-ray attenuation, while for 
magnetic resonance images (MRI) the physical property 
measured is related to various properties such as 

30 proton density. 

As shown in Fig. 2, the series of two- 
dimensional images 12 is stacked to form a three- 
dimensional volume 13, thereby defining a three- 



12 



dimensional matrix (X, Z axes) that represents at 
least one physical property associated with the three- 
dimensional structure at coordinates positioned 
throughout the three-dimensional volume. The three- 
dimensional matrix is composed of three-dimensional 
volume elements, or voxels, which are analogous to 
two-dimensional pixels. From the three-dimensional 
volume 13, a targeted volume 14 is selected for three- 
dimensional rendering. For example, the targeted 
volume 14 may include a selected organ or region of 
interest that is to be isolated from the original 
three-dimensional volume 13. The targeted volume 14 
may include an entire organ or, alternatively, may 
include an air column or volume confined within an 
organ or a lumen of the organ. 

A dataset representing the original three- 
dimensional volume 13 may be optionally subjected to a 
dataset reduction process to decrease image or spatial 
resolution or to divide the original volume into 
smaller subvolumes of data prior to isolating the 
targeted volume 14. Dataset reduction and/or 
subvolume selection are only necessary if the 
capabilities of the graphics computer 26 are 
inadequate to process the full three-dimensional 
volume 13 for effective or efficient three-dimensional 
rendering. In order to obtain three-dimensional 
constructions 17 in real time, it may be necessary to 
reduce the size of the dataset representing the three- 
dimensional volume 13 of images. For example, the 
dataset of the original three-dimensional volume 13 
may need to be reduced from an original size of 100- 
250 Megabytes to a reduced size of about 5- 
10 Megabytes. However, the amount of reduction, if 
any, may vary depending upon the size of the original 
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dataset and the processing speed and capability of the 
graphics computer 2 6 used for three-diinensionai 
rendering • 

An optional dataset reduction process 65 is 
5 generally represented in Fig. 4. When necessary, 

dataset reduction can be accomplished by reducing the 
pixel resolution, for example, from 16 bits per pixel 
to 8 bits per pixel. The reduction in pixel 
resolution, represented by step 67 in Fig. 4, 

10 decreases the image contrast in the final displayed 
three-dimensional images by reducing the number of 
shades of gray from 2** to 2*. While not preferable in 
particular applications, dataset reduction may also be 
effected by decreasing spatial resolution, as 

15 represented by step 69 in Fig. 4. For example, each 
two-dimensional image 12 stored on a matrix of 512 
pixels X 512 pixels may be reduced to a matrix of 256 
pixels X 256 pixels through spatial manipulation of 
information. 

20 Further reduction of the dataset size of the 

original three-dimensional volume 13 can be effected, 
if necessary, by selecting a subvolume of the dataset 
to be displayed, as represented by step 68 of Fig. 4- 
For example, the colon can be subdivided into the 

25 rectum, the sigmoid colon, the descending colon, the 
splenic flexure, the transverse colon, the hepatic 
flexure, the ascending colon, and the cecum. A 
separate subvolume may then be created for one or more 
of such subdivisions of the colon. 

30 Following dataset reduction, the three- 

dimensional image of the organ or region of interest 
is segmented (isolated) from the volume of data. A 
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range of tissue values, which depends on the physical 
property measured (e.g., x-ray attenuation), may be 
selected to designate the organ or region of interest. 
The organ or region of interest is then isolated from 
5 the volume of data,. A general method or process 70 
for segmenting an image of a selected organ or region 
of interest is represented in Fig. 5. The region of 
interest may, for example, be the air column 
comprising the colon or tracheobronchial airways, or 
10 some other body part having a lumen filled with a 

homogeneous substance (i.e., air, blood, urine, etc.). 

Alternatively, the region of interest may be a section 

of bone. 



The segmentation process 70 may be effected, 
for example, by designating a range of physical 
property values bounded by threshold limits that 
function to designate the organ of interest. For the 
selected volume or subvolume, voxels falling within a 
selected thresholding range are assigned a single 
value, for example, 255 to provide a white color, 
whereas voxels falling outside of the selected 
thresholding range are assigned a different single 
value, for example, 0 to provide a black color. 
Accordingly, the selected volume is thresholded by 
comparing each voxel in the selected volume to the 
threshold limits and by assigning the appropriate 
color value 0 or 255 to each such voxel depending on 
whether each such voxel falls inside or outside the 
threshold range defined by the threshold limits. By 
thresholding the selected volume, the target volume 14 
is formed having equal voxel values. More 
specifically, a target organ 14 is produced having a 
white color while all volumes outside the threshold 
limits are produced having a black color. With the 
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exception of various artifacts, which can be 
eliminated using image-processing techniques, only the 
thresholded target organ 14 is colored white and 
everything else is colored blaclc. 

5 From the thresholded target volume 14, an 

isosurface 15 is defined. The isosurface 15 is an 
exterior surface of equal voxel value on the 
thresholded target volume 14 which is established by 
the selected thresholding range. 

10 A wireframe model 16 is generated from the 

isosurface 15. The wireframe model 16 is formed as a 
series of polygonal surfaces that approximates the 
surface of the region of interest such as the selected 
organ. The wireframe model 16 is defined by a series 

15 of vertices which are interconnected by a set of line 
segments. The wireframe model 16 appears as a three- 
dimensional wire mesh object which can be rendered 
into a three-dimensional image 17. The three- 
dimensional image 17 is generated by appropriately 

20 shading the polygons of the wireframe model 16 to 
provide a three-dimensional image of the selected 
organ. The three-dimensional image 17 is displayed on 
a computer monitor 28. Additionally, the displayed 
imagery 17 can be recorded on a video recorder 30 or 

25 photographed for future viewing. An input, in the 
form of a computer mouse 27, is provided on the 
graphics computer 26 to permit a user to manipulate 
the displayed imagery. 

The method of the present invention can be 
30 used to display a colon in three dimensions and, in a 
more specific application, to enable real-time or 
interactive three-dimensional rendering of the colon 
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which thereby enables user interaction with the colon 
imagery, i.e. virtual reality. While user interaction 
with the three-dimensional images of the colon is 
simulated, the three-dimensional image itself is an 
5 accurate, three-dimensional view of the actual colon. 

A method for generating interactive, three- 
dimensional renderings of a patient's colon in 
accordance with the present invention is generally set 
forth in Fig. i. At step 40, a patient is initially 

10 prepared for imaging by cleansing the patient's colon. 
The cleansing procedure can be accomplished with a 
clear liquid diet in conjunction with laxatives. 
Alternatively, a Golytely prep can be administered on 
the day before the exam. The purpose of the cleansing 

15 procedure is to eliminate feces from the colon. 

Optimally, an absolutely clean colon is desired prior 
to computed tomography (CT) scanning. Any retained 
feces or fluid can simulate or mask small polyps 
because it is sometimes difficult to differentiate 

20 feces from the colon wall. The effectiveness of using 
a clear liquid diet or a Golytely prep procedure may 
still be somewhat hindered by small amounts of 
retained feces. As an alternative to cleansing the 
colon, or in conjunction with cleansing the colon, the 

2S patient can be fed a low residue diet combined with a 
contrast agent (such as a low density barium, for 
example, 1.5% W/V barium) for about three days. Such 
a procedure may serve to homogeneously opacify any 
retained stool so that the image of the feces can then 

>0 be subtracted from the final display, or at least from 
selected images, using image processing techniques. 

Once the colon has been cleansed, the colon 
is insufflated with gas to distend the colon. 
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Distending the colon assures that the interior surface 
of the colon will be clearly visible in the final 
image display. A rectal catheter, i.e., a standard 
barium enema catheter (0.5 inch (1.27 cm) diameter is 
5 inserted and gas is introduced into the colon ♦ The 
colon is filled to a predetermined pressure or volume 
by introducing the gas either as discrete puffs or at 
a constant flow rate. Although air can be used, CO^ 
may be preferred since COj passes through the colonic 
10 mucosa into the bloodstream and is subseguently 

exhaled, thereby decreasing the amount of bloating and 
cramping experienced by a patient after the 
examination. Unlike conventional colonoscopy, there 
is no need to sedate the patient for this procedure. 

15 After insufflation, the colon is then 

scanned, at step 45 of Fig. 1, by a helical CT scanner 
22 to produce a series of two-dimensional images 12 of 
the colon. The picture elements, or pixels, in the 
images 12 represent at least one physical property 

20 associated with the colon. The physical property, for 
example, may be the x-ray attenuation of the colon 
wall or of the air column within the colon. The 
images 12 are generally taken at regularly spaced 
locations throughout the abdominal region of the 

25 patient. The smaller the spacing between successive 
images 12, the better the resolution in the final 
displayed imagery. Preferably, the spacing between 
successive images 12 is approximately 1mm to produce 
isocubic voxels since the X and Y dimensions are each 

30 approximately 1mm. 

Immediately after inflating the colon, the 
abdomen is scanned, for example, with a GE HiSpeed 
Advantage Helical CT Scanner 22, during a single 
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breath-hold acquisition which is completed in about 
3 0-50 seconds. The scanning parameters may consist of 
a 0.20 inch (5 mm) x-ray beam collimation, 
0*4 inch/ sec (10 mm/sec) table speed to provide 
5 2:1 pitch, and a 0.04 inch (1 mm) image reconstruction 
interval. The x-ray beam collimation is the width of 
the x-ray beam which thereby establishes the CT slice 
thickness and Z-axis spatial resolution. The pitch is 
the CT table speed divided by the collimation. The 

10 image reconstruction interval reflects the interval at 
which two-dimensional images are reconstructed. Using 
an x-ray beam collimation of 5mm and a selected 
reconstruction interval of 1mm, images reflecting a 
5mm CT slice thickness are generated at 1mm intervals. 

15 Consequently, there is an overlap of 4mm between 

successive 5mm thick images at a 1mm reconstruction 
interval. 



A 50 second scan at a table speed of 10mm 
per second in a Z-axis direction creates a volume of 

2 0 data having a Z-axis length of 500mm. Using a imm 

reconstruction interval over the length of the Z-axis 
of the volume of data produces 500 images with each 
image representing a 5mm thick section along the Z- 
axis. Consequently, up to 500 images may need to be 

25 stored in compressed format on the CT scanner console 
or on a GE Independent Console (physician computer 
console) 24 associated with the scanner 22. After 
completion of the CT scan, the rectal catheter is 
removed, the gas is expelled, and the patient is 

30 discharged. 

The set of CT images 12 consisting of up to 
500 images is then extracted, at step 50 of Fig. 1, 
from a database on the computer console 24 in 
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compressed format. Once the data has been extracted, 
the data is transferred from the console 24 at step 52 
of Fig. 1/ over a fiberoptic network 25, to a graphics 
computer work station 26, such as a Crimson VGXT 
5 computer work station (150 MHz processor, 256 Mbytes 
RAM) from Silicon Graphics, Inc. (SGI, Mountain View, 
CA) . The image files 12 are preferably transferred in 
the compressed format and then decompressed on the SGI 
graphics computer 26. Alternatively, the image files 
10 can be decompressed on the G£ computer console 24 and 
then transferred to the SGI graphics computer 26. The 
fiberoptic network 25 may comprise an ethernet 
network, asynchronous transfer mode (ATM) network, or 
a Fiber Distributed Data Interface (FDDI) network. 

15 The extraction and transfer processes, 50 

and 52, are performed by three program modules. To 
perform the extraction process 50, the first module 
residing on the computer console 24 extracts the 
images one at a time from the image database on the 

20 computer console and places each extracted image file 
in a subdirectory on the computer console 24. In 
addition to the CT image files, a text file containing 
information about the patient and the type of case 
(e.g. colon) is placed in the subdirectory on the 

25 computer console 24 so that the extracted image files 
are properly correlated with the appropriate patient 
and type of case. The second module, which resides on 
the graphics computer 26 is initiated every 5 minutes. 
The purpose of the second module is to transfer the 

30 text file and corresponding image files from the 

computer console 24 to the graphics computer 26 and to 
delete such files from the computer console. The text 
file and image files are stored in a temporary 
directory on the graphics computer 26. The third 
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module is also initiated every 5 minutes and is 
interleaved with the second module. The third module 
determines if all of the files associated with a 
specific patient have been transferred to the graphics 
computer. If all of the files of the specific patient 
have been transferred, the third module then organizes 
the transferred files in a patient subdirectory on the 
graphics computer 26 according to the case type. The 
entire process generally takes about 1 hour and is a 
rate limiting step- The image transfer time can be 
reduced, however, by using the DICOM 3 image standard 
for data transmission. 



Once the data transfer to the graphics 
computer 26 is complete, the compressed image data is 
then decompressed, at step 55 of Fig. 1. The 
decompression of the image data is effected in 
accordance with a decompression scheme that is 
dependent on the specific compression formula used to 
originally compress the data. Different compression- 
decompression formulas may be used. After 
decompression is complete, a volume of data 13 is then 
formed at step 60 of Fig. 1 by stacking the series of 
CT images 12 in computer memory. The formation of the 
volume of data 13 can be performed by using existing 
volume formation programs, such as VoxelView^ 
(Vitallmages, Fairfield, lA) , or by using a customized 
program. Since each CT image 12 is approximately 
0.5 megabytes in size, 500 images equates to about 
250 megabytes. Therefore, a graphics computer 2 6 with 
sufficient memory, such as 256 Mbyte RAM, and adequate 
disc storage, such as greater than 1 gigabyte, is 
required. 
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Since rendering speed is inversely 
proportional to the size of the dataset, it is often 
necessary to reduce the size of the dataset of the 
three-dimensional volume 13 in order to perform three- 
5 dimensional rendering in real time. The process of 

dataset reduction is shown at step 65 of Fig. 1 and is 
illustrated in greater detail in Fig. 4. In the 
present application, the dataset is reduced from its 
original size, such as 250 megabytes, to a reduced 

10 size, such as 5-10 megabytes to expedite three- 
dimensional rendering. The dataset reduction may be 
partially accomplished by reducing the pixel 
resolution, for example, from 16 bits/pixel to 
8 bits/pixel, as shown at step 67. The reduction in 

IS pixel resolution reduces the contrast in the final 
three-dimensional image by reducing the number of 
shades of gray from 2** or 65,536 shades to 2* or 256 
shades. In the field of radiology, the gray-scale 
shading of CT images correlates to x-ray attenuation 

20 which is measured in Hounsf ield units (HU) , and which 
ranges in value from -1024 to +3072. Water is 
assigned a value of 0 HU, soft tissue falls between 20 
and 200 HU, contrast enhanced blood is > 125 HU, bones 
are > 250 HU, and air is less than -300 HU. In 

25 specific applications, the entire range of Hounsf ield 
units may not be needed. Accordingly, a selected 
region of the scale may be used. For example, the 
upper and lower values of the region may be clipped 
prior to scaling. Since the regions above 300 HU and 

30 below -700 HU do not contain useful information for 

specific applications in rendering colons and airways, 
only the region values between 300 and -700 HU are 
scaled to the 256 shades of gray. The scaling is 
linear. However, non-linear scaling could be used to 
35 emphasize a particular region. 
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Further reduction of the dataset size can be 
effected by selecting a subvolune of the dataset to be 
displayed, as represented at step 68 of Fig. 4. 
Generally, the colon is subdivided into the rectum, 
5 the sigmoid colon, the descending colon, the splenic 
flexure, the transverse colon, the hepatic flexure, 
the ascending colon, and the cecum. A separate 
subvolume may be assigned to selected subdivisions or 
sections of the colon. As shown in Fig. 7, a selected 

10 subvolume 114 containing the entire colon and portions 
of the small bowel is specified by determining a set 
of minimum and maximum coordinates (X„i„, X^, Y„i„, 
2fl.ift/ and Z^) such that the selected subvolume 114 is 
defined by the selected coordinates. For example, 

15 vertex 125 indicates X^, Y„i„, and Z^,^ and vertex 126 
indicates x^, Y„„,, and Z^. Using subvolumes reduces 
the size of the dataset and enables separate three- 
dimensional renderings of each selected subvolume. 

If further dataset reduction is desired, the 
20 dataset size can be decreased at step 69 of Fig. 4 by 
reducing the spatial resolution of the three- 
dimensional volume, for example, in the case of 500 
images, from 512 x 512 x 500 voxels to 256 x 256 x 250 
voxels. Since, reducing the spatial resolution can 
25 blur the final three-dimensional image, decreasing the 
spatial resolution is generally not a preferred 
process for dataset reduction. 

In general, the data reduction process 65 
converts the initial volume of original CT images into 
30 a transformed volume made of reduced CT images. The 
transformed volume is thereafter used to permit 
isolation of the targeted organ or other area of 
interest. The reduction of the dataset size may not 
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be necessary, however, if the graphics computer 26 has 
sufficient memory and processing capability relative 
to the size of the initial volume of original CT 
images to effect efficient three-dimensional 
S renderings. 

In order to produce a three-dimensional 
rendering of only the colon, the colon must be 
isolated from the volume of data by image 
segmentation, as generally represented at step 70 in 

10 Fig. 1 and as represented in greater detail in Fig. 5. 
Image segmentation can be performed before or after 
dataset reduction. A volume file pertaining to a 
patient is selected at step 71 of Fig. 1 and read at 
step 72 into the active RAM memory of the computer 26. 

15 An optional Sample Crop procedure 73 for subcropping 
and/or subsampling the volume of data can be invoked 
to further reduce the volume of the dataset if 
desired. The Sample Crop procedure 73 enables the 
user to crop the volume of data along the X-axis, the 

20 Y-axis and the Z-axis and further permits the user to 
subsample data, if desired, along each axis in order 
to further reduce the volume dataset. After the 
Sample Crop procedure 73, an Orthoslice procedure, as 
represented at step 74 in Fig. 5, is utilized to 

25 select a slice through the volume. More specifically, 
an orthogonal slice through the volume is taken along 
the axial plane (a cross-sectional (X-Y) plane through 
the body perpendicular to the Z-axis and to the 
patient's spine), the coronal plane (a side-to-side 

30 (X-Z) plane through the body normal to the Y-axis) , or 
a sagittal plane (a front-to-back (Y-Z) plane through 
the body normal to the X-axis) . 
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The particular orthogonal plane and the 
specific location of the slice plane may be selected 
by the user. Preferably, the orthoslice is selected 
to pass through a complex portion of the anatomy. The 

5 selected orthoslice is displayed at step 76 on the 

display monitor 28 of the graphics computer 26. After 
the orthoslice is selected and displayed, a 
thresholded version of the same image is also 
displayed on the monitor 28. A threshold range is 

0 used at step 75 to define a range of x-ray attenuation 
values that represent the organ of interest. The 
region of interest may be the air column which 
designates the air and soft tissue interface of the 
colon wall. 



The specific value of each pixel in the 
orthoslice corresponds to a physical property such as 
the x-ray attenuation at the location of each such 
pixel. To effect thresholding of the orthoslice, each 
individual pixel or grid position in the orthoslice is 
compared, at step 75, to the selected threshold range 
to determine if each such pixel value lies in the 
designated threshold range. If an input pixel from 
the orthoslice falls within the thresholded range, 
such input pixel is set to a selected maximum value, 
e.g. a value of 255 corresponding to a white color, in 
the thresholded image. Otherwise, the input pixel is 
set to a selected minimum value, e.g. a value of 0 
corresponding to a black color, in the thresholded 
image to indicate that such input pixel from the 
orthoslice falls outside the designated threshold 
range. 



The threshold range can be deduced by the 
physician based on his/her experience as well as a 



%VO 96/13207 PCT/US95/14022 



visual comparison of the orthoslice with the 
thresholded image. The thresholded image is displayed 
simultaneously with the orthoslice on display monitor 
28, as indicated at step 76 of Fig. 5, to enable the 
5 threshold range to be manually adjusted in real time 
to produce a good match between the thresholded image 
and the anatomical detail of the orthoslice. As shown 
in Fig. 8, the orthoslice is one of the reduced CT 
images 112 through the volume and it is displayed 

10 alongside a corresponding thresholded image 115. The 
thresholded image 115 is a single reduced CT image 
that has been subjected to the acceptance criterion of 
the thresholding process. The threshold range is 
varied until the thresholded image 115 closely matches 

15 the organ of interest in the orthoslice 112. 

The threshold range thus obtained by 
thresholding the orthoslice is then applied globally 
to the volume of data at step 79 of Fig. 5 to create a 
thresholded volume. An exterior surface of equal 
2 0 voxel value, or an isosurface 15, may be defined on 
the thresholded volume. 

The isosurface 15 of the thresholded volume 
is then used as the basis for generating a wireframe 
model 16 of the colon as set forth in Fig. 5. As 

25 illustrated in Fig. 9, a wireframe model 116 of the 
rectum portion of the colon is depicted in which the 
spatial resolution has been reduced from 512^ to 256\ 
The vertices of the wireframe model 116 define a 
series of polygonal surfaces 136 that approximate the 

30 surface of the organ of interest. Associated with 

each of the polygonal surfaces 136 is a vector which 
is perpendicular or normal to the surface 13 6. 
Various algorithms can be used for creating the 
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wireframe model 116 including, but not limited to, 
marching cubes, dividing cubes, and marching 
tetrahedrons. According to the marching cubes 
algorithm, the wireframe model 116 of the colon is 
assembled by fitting a polygon or polygons through 
each voxel that is deemed to sit astride the 
isosurface 15 (i.e., through each voxel on the 
isosurface 15) . The way in which the polygon or 
polygons are situated within each voxel depends upon 
the distribution of the vertices of the voxel that lie 
inside and outside the surface. It is generally 
assumed that there are 15 possible distributions. The 
final position and orientation of each polygon within 
each voxel is determined by the strength of the 
measured property at the vertices. The polygon or 
polygons are therefore an accurate representation of 
the surface as it passes through the voxel. 

Since subjecting the entire data volume to 
simple thresholding may sometimes over/under estimate 
20 the diameter of the colon and accentuate image noise, 
region growing and edge detection methods may be 
employed for segmenting the colon. A generalized 
region growing procedure 81 is set forth in Fig. 6. 
Region growing 81 can be used instead of the 
25 thresholding procedure to effect segmentation of the 
organ or selected region of interest. 

In region growing, a seed voxel is selected 
at step 82 which lies within the organ or region of 
interest. At step 83, the seed voxel is set to a 
30 first value (i.e., 255 for white). The value of a 

neighboring voxel is then read at step 84. The value 
of the neighboring voxel is compared to a threshold 
range to determine if such neighboring voxel falls 
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within the acceptance threshold range at step 85. If 
the neighboring voxel falls within the acceptance 
range, the neighboring voxel is set to the same first 
value as the seed at step 86. The process then 
5 returns to the seed voxel (i.e., neighbor-1) at step 
87. A check is made, at step 88, to determine if all 
the voxels neighboring the seed voxel have been 
tested. If all the neighboring voxels have not been 
tested, another voxel neighboring the seed voxel is 

10 read at step 84 and processing continues as described 
above. If all of the neighboring voxels have been 
tested at step 88, a new seed type voxel is picked at 
step 89. The new seed voxel is a voxel which has been 
determined to lie within the region of interest but 

15 whose neighboring voxels have not yet been tested at 
step 85. Processing then continues at step 84, as 
described above. 

Returning to step 85, if the neighboring 
voxel does not fall within the acceptance range, then 

20 the neighboring voxel is set to a second value (i.e., 
0 for black) ^ at step 61, thereby indicating that the 
neighboring voxel lies at the edge of the region of 
interest. The process then returns to the seed voxel 
(i.e., neighbor-1) at step 87 A check is then made, 

25 at step 88', to determine if all the voxels 

neighboring the seed voxel have been tested. If all 
the neighboring voxels have not been tested, another 
voxel neighboring the seed voxel is read at step 84 
and processing continues as described above. If all 

30 of the neighboring voxels have been tested, a new seed 
type voxel is picked at step 89 ' . A check is then 
made, at step 88' *, to determine if all the voxels 
neighboring the new seed type voxel have been tested. 
If all the neighboring voxels have not been tested, a 
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voxel neighboring the new seed type voxel is read at 
step 84 and processing continues as described above. 
If all the neighboring voxels have been tested, 
processing stops at step 62, thereby indicating that 
the region of interest is completely bounded by edge 
voxels. In this manner, the organ of interest can be 
detected without subjecting the entire volume of data 
to thresholding. This technique is therefore capable 
of reducing the occurrence of artifacts (e.g., air 
filled organs other than the organ of interest) in the 
final three-dimensional rendering. 

As an alternative to setting the voxel value 
to a first or second value to indicate whether or not 
the voxel lies at the edge of the region of interest, 
an unused, high order bit can be marked to indicate 
that the voxel lies at the edge. For example, if the 
voxel value is stored in the first 12 bits of a 16 bit 
variable, the last 4 bits can be used to indicate 
whether the voxel lies at the edge without altering 
the information contained in the first 12 bits. This 
allows all of the information originally contained in 
the voxel value to be used for generating a wireframe 
model . 



With simple thresholding or region growing, 
the size of the object of interest can be either 
overestimated or underestimated, depending upon the 
chosen threshold range. To avoid overestimating or 
underestimating the size of the object of interest and 
to preserve the integrity of the model, the threshold 
range determining the size of the object of interest 
can be intentionally xinderestimated and then the 
segmented object of interest can be adjusted using 
morphological dilation. Simple thresholding, as at 



29 



wo 96/13207 



PCT/US95/14022 



Step 75 in Fig. 5, or region growing, as at step 85 in 
Fig. S, can be used to underestimate the size of the 
object of interest by selecting a narrower 
thresholding range than would normally be used to 
5 segment the particular object of interest. The size 
of the segmented object of interest is then increased 
by adding a "layer" of voxels to the segmented object 
of interest using morphological dilation. This is 
accomplished by adding all the voxels in the data 
10 volume which are adjacent to the segmented object of 
interest to the segmented object of interest. 

A process for morphological dilation is 
shown in Fig. 27. The process starts at step 300. At 
step 301 a voxel is selected. At step 306 it is 

15 determined whether the voxel is marked as lying at the 
edge of the region of interest. If the voxel is 
marked as an edge voxel, then the process continues at 
step 308. If the voxel is not marked as an edge 
voxel, then the process procedes to step 310. At 

20 step 308, all of the voxels adjacent to the selected 
voxel are marked as belonging to the region of 
interest. Alternatively, a user-defined "thickness" 
of voxels can be added to the segmented object of 
interest. In such a case, all of the voxels within a 

25 specified number of voxels from the selected voxel are 
marked as belonging to the region of interest. At 
step 310 it is determined whether all of the voxels 
have been analyzed. If all of the voxels have not 
been analyzed, then the process returns to step 3 01. 

30 If all of the voxels have been analyzed, then the 
process ends at step 312. 

The geometry of the wireframe model 16 is 
stored at step 78 of Fig. 5. The wireframe model is 
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Stored in the form of a set of vertices and 
interconnecting line segments that define the 
wireframe model 16. The wireframe model 16 is then 
rendered into a three-dimensional image 17, as 
5 represented at step 80 in Figs. 1 and 5. As 

illustrated in Fig. 10, a three-dimensional image 117 
of the rectum section of the colon has been rendered 
with reduced spatial resolution from the wireframe 
model 116 depicted in Fig. 9. Images of three- 

10 dimensional objects {wireframe models) in world 

coordinates (X, Y, Z) are projected to two-dimensional 
screen coordinates using ray-tracing techniques. 
Imaginary rays sent from a user's viewpoint pass 
through a viewing plane (referenced to a monitor 

15 screen) and into the object (wireframe model) . If a 
ray intersects an object, the corresponding viewing 
plane pixel is painted with a color; if no 
intersection is found, the pixel is painted as 
background. The criteria used to stop a ray 

20 determines what value is projected onto the viewing 

plane pixel . Surface rendering as used to render the 
image of the rectum shown in Fig. 10 projects the 
first visible voxel. Volume rendering projects a 
weighted average of all voxels along a ray. 

25 Animation of three-dimensional objects is 

achieved by rapidly displaying multiple three- 
dimensional views of the volume. Surface rendering 
calculations are fast enough to allow interactive 
manipulation (i.e., virtual reality) of the volume. 

30 Additional lighting techniques enhance an object's 
features by altering the shade of color of each 
viewing plane pixel. For example, "shaded surface" 
images add an illusion of depth by coloring pixels 
closer to the camera viewpoint with lighter shades. 
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Pixels in a shaded surface image reflect the distance 
between the anatomical structure and the user's 
viewpoint (not the original voxel values) . 
Information on the relative "position" of the user and 
5 a virtual light source, along with the information 
about the wireframe model 16, are used to 
appropriately shade the wireframe model 16 to produce 
a realistic impression of the anatomy. The rendering 
process 80 can be accomplished using a general purpose 
10 volume visualization program, such as IRIS Explorer^. 

Alternatively, volume rendering can be used 
to animate three-dimensional objects. Unlike surface 
rendering which projects an image of a wireframe 
model, volume rendering projects a weighted average of 

15 all voxels along a ray. The contrast of the three- 
dimensional image produced using volume rendering can 
be enhanced by assigning to each voxel an opacity 
value. The opacity value cam be adjusted to make 
targeted anatomy opaque and undesired anatomy 

20 transparent. In one embodiment, the assigned opacity 
values are the same for all voxels. In a second 
embodiment, the assigned opacity values are set 
randomly to produce a visually meaningful image while, 
for example viewing the image. In a third embodiment, 

25 the opacity values vary as a function of voxel value. 
For example, the relation between opacity value and 
voxel value can be determined by calculating the 
inverse of the histogram curve and assigning the 
values found along this curve as opacity values. The 

30 histogram curve reflects the number of voxels of a 
given voxel value as a function of voxel value. 

The rendering step 80 occurs rapidly and 
interactively, thus giving the user the ability to 
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"fly" through the volume of data. The direction of 
"flight" is controlled by the computer mouse 27 
through directional pointing of the cursor, and the 
speed (both backwards and forwards) is controlled by 
pressing buttons on the computer mouse 27. The speed 
of interactive three-dimensional rendering produces a 
"virtual reality" environment and allows the user to 
examine the image data in a manner that is analogous 
to real endoscopy. 

The path (camera coordinates) of each 
simulated flight can be stored and/or recorded. The 
stored path can be used in a "playback" mode to 
retrace the flight path. Individual three-dimensional 
scenes (views, images) may also be recorded (stored) 
on the computer like photographs. The geometric 
representation of the wireframe model 16 of the colon 
and the volume dataset used to create the wireframe 
model are stored, at step 78 of Fig. 5, on digital 
audio tape (DAT) or, preferably, on read/write optical 
discs. Each simulated "flight" through the colon can 
be recorded on VHS videotape on a video recorder 30 
for archival purposes at step 90 of Fig. i. Each 
flight may be recorded for later review by, for 
example, gastroenterologists and surgeons. 

The recorded path can be used to allow the 
user to re -trace the path through the object of 
interest. This could assist the user if the user 
loses orientation or location during a flight through 
the object of interest. The user would back-up 
through the recorded camera coordinates in reverse 
order until the user returned to a location within the 
object of interest which the user could identify. The 
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user could then continue the flight through the object 
of interest from that camera position. 

To achieve an adequate speed of rendering 
(flying speed) but still preserve the anatomical 
5 detail of the original CT data volume 13, it is 

possible to navigate or "fly" through reduced three- 
dimensional imagery based on a wireframe model 16 
built from the reduced dataset volume. When the 
motion stops, however, the three-dimensional scene may 
10 be redisplayed with the highest resolution possible 
using the original CT image volume 13 . 

Alternatively, the user can navigate through 
the three-dimensional imagery generated using surface 
rendering, select a point as the center of a 
15 subvolume, and have a three-dimensional image of the 
subvolume rendered using volume rendering. 

In addition to allowing the user to "fly" 
through the colon, the method of the present invention 
can be used to take the user on a guided tour of the 

20 colon in an "auto-pilot" mode of operation. In the 

"auto -pi lot" mode, the user is moved at a preselected 
speed through the three-dimensional representation of 
the colon along the center line of the lumen of the 
colon. The center line of the colon can be determined 

25 in one of several ways. One method of determining the 
central path through the lumen of the colon is 
illustrated in Fig. 23. A seed point 91 is selected 
which lies within the lumen of the segmented colon 
117'''. The plane 92, passing through point 91 that 

30 has the minimum area 93 of colon dissection is 
determined and the center 95 of such area 93 is 
calculated. A new point is then selected which lies 



34 



wo 96/13207 PCT/US9S/14022 



Icm away from center point 95 in a perpendicular 
direction relative to the surface area 93 in plane 92 
as shovm by arrow 94 . A new plane of minimum area 
that dissects the colon and passes through the new 
5 point is determined and the corresponding center of 
that new area is calculated. This iterative process 
is continued until a central path connecting the 
center points is determined. 

Alternatively, repetitive morphological 
"erosions" can be performed on the segmented colon 
through an iterative process analogous to removing 
"layers" of voxels. As each layer is eroded, the 
iterative level at which voxels disappear in each such 
layer is recorded. The erosion process is continued 
until all of the voxels disappear. The central path 
through the colon is then constructed by connecting 
the highest valued voxels (i.e., the last voxels to 
disappear by erosion) . 

By determining the central path through the 
colon, a single oblique (reformatted) plane (gray 
scale image) that is perpendicular to the central path 
at a certain point can be displayed thereby allowing 
simultaneous viewing of the wall thickness of the 
colon and surrounding anatomy and/or pathology during 
the automatic flight . The oblique plane is 
perpendicular to the central path but is usually 
oblique relative to the three orthogonal planes. 

The method of the present invention can also 
be used to display the colon in a "split" open 
30 view 118, as shown in Fig. 12. A three-dimensional 

external rendering of the rectum 117' of the colon, as 
depicted in Fig. 11, can be split along the center 
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line to expose the interior surface 119 of a selected 
half of the colon. This split display mode is 
particularly advantageous when simultaneous viewing of 
extended sectional lengths of the colon is desired or 
5 when viewing of the anatomy must be done with a 

computer that is not capable of rendering with real- 
time speed {i.e., not allowing the user to "fly" 
through the colon at a sufficiently fast speed) . 

The "split" view can be generated in one of 
several ways. The first method is to define a cutting 
plane that is disposed parallel to the plane of the 
screen of the monitor 28 such that all portions of 
objects passing through the imaginary plane are made 
invisible as the objects are pulled closer to the 
viewer and intersect the cutting plane. Defining a 
cutting plane is a standard part of Explorer' s 
renderer (viewer) . The three-dimensional object 
(colon) is rotated in space and moved closer to the 
user (viewer) . As the colon passes through the 
cutting plane, the half or portion of the colon closer 
to the user becomes invisible but the distant half or 
portion, with its surface detail, is available for 
inspection. 

Another method for producing a split view is 
25 to predefine subvolumes in accordance with step 68 of 
Fig. 4 so that a selected sectional length of colon is 
contained in two separate subvolumes corresponding to 
the two separate halves of the sectional length of the 
colon. That is, two separate subvolumes can be 
30 created so that one subvolume shows one open half of 
the colon and the second subvolume shows the other 
open half. Defining the bounding points to thereby 
define a bounding subvolume may be done visually by 
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looking at orthoslices for represencing the subvolume. 
It is faster to manipulate the two-dimensional 
orthoslice images rather than the three-dimensional 
volume. Once the bounding coordinates {X^„, Y^„, Z„i„, 
5 and X^, Y^, Z^^) are defined, then the subvolume 

representing one-half of a colon section is processed 
(rendered) • 

Another desired method for splitting the 
colon is to use an imaginary cutting plane (line) that 

10 passes through the colon perpendicular to the central 
path through the lumen of the colon. The cutting line 
is maintained at a constant level parallel to the X-Z 
plane. All voxels on one side of the cutting line are 
made invisible (i.e., set to zero value or black, 

15 since voxels in the segmented colon have been 

previously set to value 255 or white) and the white 
half of the colon is rendered. Then, the process is 
reversed and the other half of the colon is rendered. 
Alternatively, the wireframe model 16 may be split in 

20 half prior to rendering. 

An additional feature of the present 
invention is that an external map view 137 of the 
selected organ can be displayed, as shown in Fig. 13, 

25 to assist the user if the user loses orientation or 
location during a flight through the lumen of the 
organ. Fig. 14 shows a three-dimensional rendering 
117'' from inside the colon. If the user requests the 
display of a map view while inside the rendering 117'' 

30 of the rectum shown in Fig. 14, the external map view 
137 of Fig. 13 will be displayed with an indicator 138 
for indicating the position of the three-dimensional 
rendering 117" of Fig. 14. The indicator 138 can be 
a as shown in Fig. 13, or any other selected 
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symbol. PreferaJDly, the indicator 138 may be in the 
form of an arrow which also indicates the line of 
sight within the three-dimensional imagery. 

Yet another feature of the present invention 
5 is that distances, areas, circumferences, and volumes 
can be measured directly from the three-dimensional 
image. To measure distances, the user would pick two 
points in the three-dimensional image and the distance 
between the points would be determined. Analogously, 

10 to measure cross -sectional areas of an object, the 

user would pick three points in the three-dimensional 
image thereby defining a plane passing through the 
object. The area of the plane bounded by the object's 
borders could then be determined. To measure volumes, 

15 the user could select a seed point within the object 

and then use region growing or region growing followed 
by morphological dilation to segment the desired 
structure. The volume of the segmented structure 
could then be determined. This feature is especially 

20 useful for monitoring the growth of lesions or polyps. 

The method of the present invention is 
particularly well suited for identifying polyps and 
growths that extend into the lumen of the colon. 
However, some precancerous growths are manifested as a 

25 subtle thickening of the colon wall. The present 

invention can also be used to identify the areas of 
thickening as well. This can be accomplished by 
calculating and displaying a two-dimensional oblique 
plane perpendicular to the central axis of the colon 

30 whose location relative to the central path 

corresponds to the three-dimensional view at hand. 
Alternatively, a segmentation process could be used to 
isolate the colon wall instead of the air column 
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within the colon to determine the wall thickness. 
Pursuant to a texture mapping procedure, thickened 
areas or other areas of pathology can be visually 
distinguished from the normal wall of the colon in the 
5 three-dimensional imagery by displaying the polygonal 
surfaces on the wireframe model that represent the 
thickened or pathological areas with a distinguishable 
color. 

The method of the present invention can also 
10 be used to display a tracheobronchial airway in three 
dimensions, in accordance with the general methods 
described in connection with Figs. 1 and 2 and with 
the system described in connection with Fig. 3. More 
specifically, a patient is initially prepared at 
15 step 40 by administering a nonionic intravenous bolus 
of iodinated contrast agent with a power injector to 
aid in distinguishing the blood vessels surrounding 
the tracheobronchial airway. After an appropriate 
time delay (approximately 70 seconds) , the patient is 
20 scanned at step 45 from the thoracic inlet to the lung 
base to produce a series of two-dimensional images 12 . 
The images 12 represent at least one physical property 
associated with the tracheobronchial airway. This 
property may be, for example, the x-ray attenuation 
25 value measured with helical CT scanning. Preferably, 
the spacing between successive images 12 is 
approximately 1 mm to produce isocubic voxels. 

The scans may be performed with a GE HiSpeed 
Advantage Helical CT Scanner 22 during a single 
30 breath-hold accjuisition which is completed in about 

30 seconds. The scanning parameters may consist of a 
0.12 inch (3 mm) x-ray beam collimation, 0.24 inch/sec 
(6 mm/sec) table speed (2:1 pitch), and a 0.04 inch 
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(1 ram) image reconstruction interval. As a result of 
the beam collimation and the reconstruction interval 
selected, there is considerable (2 mm) overlap between 
successive images 12. Typically, up to about 
5 200 images are obtained. The images are stored in a 
compressed format on a GE computer console 24 
associated with the scanner 22. 

The series of CT scans is then extracted at 
step 50 from the image database on the computer 

10 console 24 in compressed format. Each compressed 
image represents a 512 x 512 image of picture 
elements, or pixels, and each pixel is comprised of 16 
bits of data (16 bits/pixel) . Once the data has been 
extracted, the data is transferred at step 52 over a 

15 fiberoptic network 25 to the graphics computer work 

station 26, such as the Silicon Graphics Crimson VGXT 
computer work station (150 MHz processor, 256 Mbytes 
RAM) , The image files 12 are preferably transferred 
in the compressed format and then decompressed at 

20 step 55 on the graphics computer 26 . The extraction 
and transfer steps are performed by three program 
modules. The first module residing on the computer 
console 24 extracts the images one at a time from the 
image dataibase and places each extracted image in a 

25 subdirectory on the computer console 24. In addition 
to the image files, a text file containing information 
about the patient and the type of case (i.e., lung) is 
created. The second module, which resides on the 
graphics computer 26, is initiated every 5 minutes and 

30 transfers the patient's text file and image files from 
the computer console 24 to the graphics computer 26 
and deletes such files from the computer console 24. 
The third module, which resides on the graphics 
computer, is also initiated every 5 minutes and is 
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interleaved with the second module. The third module 
determines if all of the files associated with a 
patient have been transferred. If all of the files 
have been transferred, the third module organizes the 
transferred files in a patient subdirectory according 
to the case type. The entire process generally takes 
about 1 hour and is a rate limiting step. The image 
transfer time can be reduced by utilizing the DICOM 3 
image standard. Once the data transfer is complete, 
the remaining steps are performed on the graphics 
computer 26. 



A volume of data 13 is then formed at 
step 60 by stacking the series of CT images 12 in the 
computer memory. Since each CT image 12 is 
approximately 0.5 megabytes in size, 200 images 
equates to about 100 megabytes. Therefore, a machine 
with sufficient memory and adequate storage is needed. 



Since rendering speed is inversely 
proportional to the size of the volume of data to be 
rendered, it is often necessary to reduce the size of 
the dataset in order to effectively perform three- 
dimensional rendering in real time. Dataset reduction 
is generally shown at step 65 of Fig. l and, more 
specifically, in Fig. 4. In application, the dataset 
is reduced from 100 megabytes to about 5-10 megabytes. 
Dataset reduction is partially accomplished by 
reducing the pixel resolution from 16 to 8 bits/pixel, 
as represented at step 67. The reduction in pixel 
resolution reduces the contrast in the final displayed 
images by reducing the number of shades of gray to 
256. In the field of radiology, the gray scale 
shading of CT images corresponds to x-ray attenuation 
values measured in Hounsf ield units (HU) , which range 
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in value from -1024 HU to +3072 HU. Water is assigned 
a value of 0 HU, soft tissue falls between 20 and 
200 HU, contrast enhanced blood is > 125 HU, bones are 
> 250 HU, and air is less than -300 HU. Since the 
5 regions above 500 HU and below -700 HU do not contain 
information useful for rendering the bronchial 
airways, or the surrounding blood vessels and lymph 
nodes, the region between 500 and -700 HU is scaled to 
the 256 shades of gray. The scaling is linear. 
10 However, non-linear scaling could be used to emphasize 
a particular region. 

Further reduction of the dataset size can be 
accomplished by selecting a subvolume of the dataset 
to be displayed, as represented at step 68 of Fig. 4. 

15 As illustrated in Fig. 15, a selected subvolume 214 is 
depicted containing the entire tracheobronchial 
airways. The bronchial airways are isolated from the 
entire thorax. The subvolume 214 is defined, as shown 
in Fig. 15, by determining a set of minimum and 

20 maximum coordinates (X^xn' ^min' ^tm* ^tn* 2«»x) so 

that the airways are contained within the subvolume 
214 defined by the selected coordinates. For example, 
vertex 22 5 indicates coordinate X„i„, Y „i„, and Z„^i„ and 
vertex 226 indicates coordinate X^^* and Z„j»x- 

25 The dataset size can be reduced further at 

step 69 of Fig. 4 by decreasing the spatial 
resolution, for example in the case of 200 images, 
from 512 x 512 x 200 voxels to 256 x 256 x 100 voxels. 
Since reducing the spatial resolution can blur the 

30 final displayed images, decreasing the spatial 
resolution is generally not preferred. 
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The data reduction step 65 functions to 
convert the original volume of CT images into a volume 
of reduced CT images. The volume of reduced CT images 
is then used to construct the three-dimensional 
5 imagery. 

As represented at step 70 of Fig. 1, the 
bronchial airways are isolated within the volume of 
data by image segmentation. Image segmentation can be 
performed before or after dataset reduction. To 

10 effect image segmentation, the volume file pertaining 
to the patient and case of interest are selected at 
step 71 of Fig. 5 and read at step 72 into the active 
RAM memory of the computer 26. An optional 
subcropping and/or subsampling step 73 can be 

15 performed, if desired, to further reduce the size of 
the dataset. A threshold range is used to define a 
selected range of x-ray attenuation values contained 
within the organ of interest. Typically, the region 
of interest is the air column which designates the air 

20 and soft tissue interface of the bronchial wall. 

At step 74, an orthoslice is selected 
through a complex portion of the anatomy. The 
orthoslice is a cross -sectional two-dimensional slice 
(image) through the dataset parallel to one of the 

25 primary planes (X-Y, X-Z, or Y-Z) . The orthoslice is 
displayed on the monitor at step 76 as one of the 
reduced CT images through the volume. Alongside the 
reduced CT image, a thresholded image is also 
displayed at step 76. The displayed thresholded image 

30 represents the orthoslice of the reduced CT image that 
has been subjected to an acceptance criterion of the 
thresholding process as previously described. As 
shown in Fig. 16, an orthoslice image 212 of the 
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airways is simultaneously displayed with a 
corresponding thresholded image 215 . The thresholded 
image 215 in Fig. 16 is shown with the region of 
interest, the air column within the airways, in black . 
5 The threshold range is varied until the displayed 
thresholded image 215 best matches the anatomical 
detail of the organ of interest in the displayed gray 
scale orthoslice image 212, The threshold range thus 
obtained is applied throughout the selected volume of 

10 data to create a thresholded volume representing the 
airways. As an alternative to thresholding, a region 
growing procedure, as previously described and as 
represented in Fig. 6, may be employed to isolate the 
organ or region of interest. The region growing 

15 procedure is able to reduce the occurrence of 

artifacts (i.e., air filled structures other than the 
organ of interest) in the three-dimensional imagery. 

An isosurface 15 of the thresholded volume 
is then used as the basis for forming a wireframe 

20 model 16 of the bronchial airways. As shown in 

Fig. 17, a wireframe model 216 of the tracheobronchial 
airways is depicted. The vertices of the wireframe 
model 216 define a series of polygonal surfaces 236 
that approximates the surface of the organ of 

25 interest. Various algorithms can be used for creating 
the wireframe model 216 including, but not limited to, 
marching cubes, dividing cubes, and marching 
tetrahedrons . 

The wireframe model 216 is rendered at step 
30 80 of Fig. 1 into an interactive, three-dimensional 

display 217, as illustrated in Fig. 18. The rendering 
procedure can be accomplished using a general purpose 
volume visualization program, such as IRIS Explorer^. 
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The rendering procedure gives the user the ability to 
"fly" through the volume of data. The direction of 
"flight" is controlled by the orientation of the 
computer mouse. The speed of the "flight" (both 
backwards and forwards) is controlled by pressing 
computer mouse buttons. Information on the relative 
"position" of the user and a virtual light source, 
along with the information from the wireframe 
model 216, are used to appropriately shade the 
wireframe model 216 to give a realistic impression of 
the anatomy. Interactive three-dimensional rendering 
creates a perception of "virtual reality", and allows 
the user to examine the CT data in a way which is 
analogous to real bronchoscopy. 

Each "flight" through the bronchial airways 
can be recorded on VHS videotape at video recorder 30, 
for archival purposes, as represented at step 90 of 
Fig. 1, and for later review by, for example, 
bronchoscopists and surgeons. In addition, each 
flight through the airways can be recorded on the 
computer 26 by storing the path of movement as a set 
of coordinates reflecting the flight path through the 
three-dimensional imagery. For example, the 
appropriate coordinates of the path of movement may be 
stored in computer memory as the path of movement is 
generated. The coordinates of the path of movement 
may be stored in a setting file on the computer 26 so 
that the path of movement may be reproduced (replayed) 
on the computer 26 at a later time. Individual three- 
dimensional scenes (views, images) may also be 
recorded (stored) on the computer like photographs. 
The wireframe computer model of the bronchial airways 
can also be stored as represented at step 78 in Fig. 5 
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on digital audio tape (DAT) or, preferably, on 
read/write optical discs. 

As an additional feature, selected "go to" 
points (three-dimensional views) may be generated and 
5 stored during a flight through the lumen of the organ. 
A "go to" procedure, generally designated 130, is 
represented in the flow chart of Fig. 25. During a 
flight through the organ, selected points of interest, 
such as three-dimensional scenes or views, may be 
10 recorded and assigned, at step 131, to buttons 
appearing on a general diagrammatic map of the 
particular organ being examined. The "go to" points 
on the map are stored, at step 132, in an appropriate 
setting file on the computer 26. In order to use the 
15 "go to" points, the user requests the display of the 
map of the organ at step 133. The "go to" points 
(buttons) appear on the map of the organ so that the 
user can move the mouse cursor 'to a selected button, 
at step 134, and then click a mouse button so that the 
20 displayed three-dimensional view is transformed to the 
chosen view at step 135. In application with the 
colon imagery, separate "go to" points may be assigned 
to respective sections corresponding to the rectum, 
sigmoid colon, descending colon, splenic flexure, 
25 transverse colon, hepatic flexure, ascending colon and 
cecum. For the tracheobronchial airways, separate "go 
to" points may be assigned to respective sections 
corresponding to the trachea, carina, right upper lobe 
bronchus, right middle lobe bronchus, right lower lobe 
30 bronchus, left upper lobe bronchus, and left lower 
lobe bronchus . 

The system 20 also provides the user with 
the ability to display the three orthogonal slices 
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(along the axial, coronal, and sagittal axes) which 
pass through any selected "pick point" in the three- 
dimensional imagery. This is accomplished by pointing 
the computer mouse cursor at the desired location in 
5 the three-dimensional imagery and "clicking" on that 
point with a computer mouse button. The point of 
intersection with the first visible surface in the 
three-dimensional imagery is captured. The three- 
dimensional coordinate (X,Y,Z position) of the point 

10 of intersection is used to calculate the three 

orthogonal planes (axial, sagittal, coronal) that pass 
through that point. The axial plane is parallel to 
the X-Y axes, the sagittal plane is parallel to the Y- 
Z axes, and the coronal plane is parallel to the X-Z 

15 axes. As shown in Fig. 26, the three orthogonal 

planes passing through a selected pick point in the 
tracheobronchial airways are displayed in three 
separate windows (140, 141, and 142) together with a 
main window 143 containing the three-dimensional image 

20 217''' of the tracheobronchial airways. Cross-hairs 

are overlaid on each orthogonal plane image indicating 
the position of the pick point. By moving the mouse 
cursor over one of the orthogonal images, the cross- 
hairs in that image follow the position of the cursor. 

25 As the position of the cross-hair changes, the other 
two orthogonal images and their cross -hairs change to 
reflect the new position of the point in three- 
dimensional space. The three-dimensional image in the 
main window remains frozen during this time and does 

30 not change. The usefulness of this function is that 
it allows a physician to view and pan through the 
reduced volume CT image data that surrounds the pick 
point. In using this process, wall thickness at a 
selected location may be determined. In addition, 

35 surrounding anatomical structure or pathology may also 

47 



wo 96/13207 



PCmJS95/14022 



be viewed. In addition to "clicking" on a single pick 
point in the three-dimensional imagery, the user may 
activate a selected combination of mouse buttons while 
moving the mouse through the three-dimensional 
5 imagery. The points of intersection are immediately 
calculated and the corresponding orthogonal planes are 
displayed. This mouse dragging procedure is analogous 
to "dragging" you finger across the surface of the 
anatomy and displaying the orthoslices interactively. 

10 Alternatively, a pick point in the center of the 
three-dimensional image can be automatically 
determined and the corresponding orthogonal planes 
displayed, with the orthogonal planes being 
continually updated as the user moves through the 

15 three-dimensional image. 

The method of the present invention also 
provides the ability to render portions of a region of 
interest transparent or semi-transparent. Typically, 
the tracheobronchial airway can be rendered semi- 

20 transparent or completely transparent to allow the 

user to "see" beyond the wall of the airway and locate 
various anatomical structures, such as lymph nodes and 
blood vessels. This aspect of the present invention 
is particularly useful as a guide for Transbronchial 

25 Needle Aspiration (TBNA) . For example, needle 

placement can be guided to a selected location through 
the tracheobronchial wall. Making the 
tracheobronchial wall transparent to reveal 
surrounding blood vessels and lymph nodes may serve to 

30 prevent unintentional puncture of blood vessels and 
lymph nodes during needle insertion. 

Referring to Fig. 24, a procedure 100 for 
making a selected organ transparent for the purpose of 
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revealing surrounding organs is depicted. After a 
volume of reduced CT images is generated at the 
selected area of interest, selected organs, such as 
the tracheobronchial airway, the surrounding blood 
5 vessels and the surrounding lymph nodes, are segmented 
from the volume of reduced CT images. At step 70', 
the air column representing the tracheobronchial 
airway is segmented. At step 70'', the blood vessels 
are segmented and at step 70''' the lymph nodes are 
10 segmented. The order in which selected organs are 

segmented may be varied. In addition, each organ may 
be segmented using a thresholding procedure or the 
region growing technique. 



If thresholding is employed for organ 
15 segmentation, the air column representing the 

tracheobronchial airway may be assigned a range less 
than -300 HU. The soft tissue representing the lymph 
nodes may be thresholded in the range of 30 to 80 HU. 
The blood vessels may be thresholded according to the 
20 contrast agent contained within the blood so that the 
blood vessels are thresholded in the range greater 
than 125 HU. 

After the appropriate wireframe models 16 
are generated for the respective organs, the airways, 
25 blood vessels and lymph nodes are rendered together in 
the same three-dimensional scene at step 80' of Fig. 
24 for display on the screen of the monitor. To 
better distinguish between respective organs, each 
wireframe model may be produced in a separate color. 



30 At step 101 of Fig. 24, one of the organs is 

selected for transparency. Then, at step 102, the 
degree of transparency is selected. The degree of 
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transparency may be selected by a sliding scale from 0 
to 100%. In application, the walls of the 
tracheobronchial airway may be selected to be 
displayed transparently, therefore the view from 
5 within the lumen of the tracheobronchial airway 

enables the location of the surrounding blood vessels 
and lymph nodes to be seen. 

The method of the present invention can also 
be used to display the tracheobronchial tree in a 
10 "split" view 218, as shown in Fig. 20. A three- 
dimensional rendering 217' of a selected section of 
the tracheobronchial tree, as shown in Fig. 19, can be 
split along the center line to expose the interior 
surface 219 of the tracheobronchial tree. 



15 An additional feature of the present 

invention is that an external map view 237 can be 
displayed as shown in Fig. 21. Fig. 22 shows a three- 
dimensional rendering 217'' from inside the 
tracheobronchial tree entering the left mainstem 

20 bronchus. Fig. 21 shows an external map view 237 with 
an indicator 238 for indicating the position within 
the tracheobronchial tree from which the rendering 
217'' in Fig. 22 is taken. The indicator 238 can be a 
as shown in Fig. 21, or any other symbol, such as 

25 an arrow which also indicates the line of sight within 
the three-dimensional imagery. 



The present invention is not limited to the 
use of images obtained from CT scanners. The present 
invention works equally well with tomographic x-ray 
30 data, ultrasound, positron emission tomography, 

emission computed tomography, and magnetic resonance 
images. In addition, although the present invention 
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has been discussed in reference to its usefulness in 
examining the colon and the tracheobronchial tree, the 
present invention is also useful in rendering three- 
dimensional images of other passageways and anatomical 
5 structures, such as arteries, veins, solid viscera, 

stomachs, bladders, bone, and ventricles of the brain. 



It will be recognized by those skilled in 
the art that changes or modifications may be made to 
the above -described embodiments without departing from 

10 the broad inventive concepts of the invention. It 

should therefore be understood that this invention is 
not limited to the particular embodiments described 
herein, but is intended to include all changes and 
modifications that are within the scope and spirit of 

15 the invention as set forth in the claims. 
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What is claimed is: 

1. A method for interactively displaying three- 
dimensional structures comprising the steps of: 

A. forming a three-dimensional volume of 
S data from a series of two-dimensional 

images representing at least one 
physical property associated with a 
three-dimensional body; 

B. segmenting a selected region of 

10 interest from the volume of data based 

on selected values of the physical 
property representing the selected 
region of interest; 

C. rendering the segmented region of 
15 interest in an interactive three- 
dimensional display. 

2. The method as recited in claim 1 comprising a 
reduction step of reducing the size of the volume of 
data before rendering the image of the segmented 

2 0 region of interest in the interactive three- 
dimensional display. 

3. The method as recited in claim 2 wherein the 
reduction step comprises reducing pixel resolution. 

4. The method as recited in claim 3 wherein the pixel 
25 resolution is reduced to 8 bits per pixel. 

5. The method as recited in claim 2 wherein the 
reduction step comprises reducing spatial resolution. 
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6. The method as recited in claim 5 wherein the 
spatial resolution is reduced to 256 pixels by 
256 pixels. 

7. The method as recited in claim 2 wherein the 

5 reduction step comprises creating a subvolume of data. 

8. The method as recited in claim 1 wherein the 
series of two-dimensional images comprises images 
taken at regularly spaced grid locations within the 
body. 

10 9. The method as recited in claim 8 wherein the 

spacing between successive grid locations is selected 
to produce isocubic voxels for the three-dimensional 
display. 

10. The method as recited in claim 1 wherein the 
15 physical property includes x-ray attenuation. 

11. The method as recited in claim 1 wherein the 
series of two-dimensional images comprises images 
taken at 1 mm regularly spaced grid locations within 

20 the body. 

12. The method as recited in claim 1 wherein the 
series of two-dimensional images comprises computed 
tomography images. 

13. The method as recited in claim 12 wherein the 
25 computed tomography images comprise helical computed 

tomography images. 
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14. The method as recited in claim 1 wherein the 
series of two-dimensional images comprises x-ray 
images. 

15. The method as recited in claim 1 wherein the body 
5 comprises a human body. 

16. The method as recited in claim 1 wherein the 
selected region of interest comprises an air column 
within a body organ. 

17. The method as recited in claim 16 wherein the air 
10 column comprises a colon. 

18. The method as recited in claim 16 wherein the air 
column comprises the bronchi. 

19. The method as recited in claim 16 wherein the air 
column comprises a larynx. 

15 20. The method as recited in claim 16 wherein the air 
column comprises a trachea. 

21. The method as recited in claim 1 wherein the 
region of interest comprises an air-tissue interface 
at a body organ . 

20 22. The method as recited in claim 1 wherein the step 
of segmenting the selected region of interest 
comprises a thresholding step for determining 
threshold values corresponding to the selected values 
of the physical property to isolate the selected 

25 region of interest- 
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23. The method as recited in claim 22 including 
forming an isosurface wireframe model of the isolated 
selection region of interest. 

24. The method as recited in claim 2 3 wherein the 
5 step of rendering an image includes the step of 

rendering a three-dimensional display of the wireframe 
model. 

25. The method as recited in claim 22 wherein the 
thresholding step comprises: 

a. a threshold selection step for 
selecting a threshold range 
corresponding to the selected values of 
the physical property representing the 
selected region of interest; and 

b, a threshold adjustment step for 
adjusting the threshold values to 
provide the threshold range for 
isolating the selected region of 
interest. 

20 26. The method as recited in claim 25 wherein the 
threshold adjustment step comprises: 

a. an orthoslice step of providing an 

orthoslice through the volume of data; 
and 

25 b. a display step for displaying the 

orthoslice and a corresponding 
thresholded image of the orthoslice, so 
that the threshold values can be 
adjusted while comparing the 

30 thresholded image to the corresponding 

orthoslice. 
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27. A method for interactively displaying a colon in 
three dimensions comprising: 

a. an image acquisition step wherein a 
series of two-dimensional images 

5 representing at least one physical 

property associated with the colon is 
acquired; 

b. a volume formation step wherein a 
three-dimensional volume of data is 

10 formed by stacking the series of two- 

dimensional images in computer memory; 

c. a segmentation step wherein a region of 
interest related to the colon is 
segmented from the volume of data based 

15 on selected values of the physical 

property representing the region of 
interest; and 

d. a rendering step wherein the segmented 
region of interest is rendered in an 

20 interactive three-dimensional display. 



28. The method as' recited in claim 27 comprising 
cleansing the colon before the image acquisition step. 

29. The method as recited in claim 27 comprising 
inflating the colon with gas before the image 

25 acquisition step. 

30. The method as recited in claim 27 wherein the 
series of two-dimensional images comprises images 
taken at regularly spaced grid locations within the 
body. 

30 31. The method as recited in claim 30 wherein the 

spacing between successive grid locations is selected 
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to produce isocubic voxels for the three-dimensional 
display. 

32. The method as recited in claim 30 wherein the 
series of two-dimensional images comprises images 

5 taken at 1 mm regularly spaced grid locations within 
the body. 

33. The method as recited in claim 27 wherein the 
physical property includes x-ray attenuation. 

34. The method as recited in claim 27 wherein the 
10 series of two-dimensional images comprises computed 

tomography images. 

35. The method as recited in claim 34 wherein the 
computed tomography images comprise helical computed 
tomography images . 

15 36. The method as recited in claim 27 wherein the 

region of interest comprises an air-tissue interface. 

37. The method as recited in claim 27 further 
comprising a data reduction step for reducing the size 
of the volume of data before rendering the segmented 

20 region of interest. 

38. The method as recited in claim 37 wherein the 
reduction step comprises reducing pixel resolution. 

39. The method as recited in claim 38 wherein the 
pixel resolution is reduced to 8 bits per pixel. 
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40. The method as recited in claim 37 wherein the 
reduction step comprises reducing spatial resolution. 

41. The method as recited in claim 40 wherein the 
spatial resolution is reduced to 2 56 pixels by 

5 256 pixels. 

42. The method as recited in claim 37 wherein the 
reduction step comprises creating a subvolume of data. 

43. The method as recited in claim 42 wherein the 
subvolume is a subvolume containing at least one of 

10 the rectum, the sigmoid colon, the descending colon, 

the splenic flexure, the transverse colon, the hepatic 
flexure, the ascending colon, and the cecum. 

44. The method as recited in claim 37 wherein the 
size of the volume of data is reduced to less than 

15 about 10 megabytes. 

45. The method as recited in claim 27 wherein the 
rendering step comprises simulating movement along a 
line which passes along the center of the lumen of the 
colon. 

20 46. The method as recited in claim 27 wherein the 

rendering step comprises splitting the colon open in 
lengthwise sections and displaying the split open 
colon so that inner surfaces of the split open colon 
are visible. 

25 47. The method as recited in claim 27 wherein the 
segmentation step comprises a step of determining 
colon wall thickness. 
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48. The method as recited in claim 47 wherein the 
step of determining the colon wall thickness comprises 
displaying a cross-sectional image through the colon. 

49. The method as recited in claim 47 wherein the 
step of determining colon wall thickness comprises a 
step of identifying a thickened section of the colon 
by visually differentiating a thickened section on a 
displayed image from normal thickness of the colon 
wall. 

50. A method for interactively displaying a 
tracheobronchial airway in three dimensions 
comprising: 

a. an image acquisition step wherein a 
series of two-dimensional images 
representing at least one physical 
property associated with the 
tracheobronchial tree is acquired; 

b. a volume formation step wherein a 
three-dimensional volume of data is 
formed by stacking the series of two- 
dimensional image in computer memory; 

c. a segmentation step wherein a region of 
interest is segmented from the volume 
of data based on selected values of the 
physical property representing the 
region of interest; and 

d. a rendering step wherein the segmented 
region of interest is rendered in an 
interactive three-dimensional display. 

51- The method as recited in claim 50 wherein the 
image acquisition step comprises a patient preparation 
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step wherein a patient is administered a nonionic 
intravenous iodinated contrast bolus. 

52. The method as recited in claim 50 wherein the 
series of two-dimensional images comprises images 
taken at regularly spaced grid locations within the 
body. 

53. The method as recited in claim 52 wherein the 
spacing between successive grid locations is selected 
to produce isocubic voxels for three-dimensional 
display. 

54. The method as recited in claim 52 wherein the 
series of two-dimensional images comprises images 
taken at 1 mm regularly spaced grid locations within 
the body. 

55. The method as recited in claim 52 wherein the 
physical property includes x-ray attenuation. 

56. The method as recited in claim 52 wherein the 
series of two-dimensional images comprises computed 
tomography images. 

57. The method as recited in claim 56 wherein the 
computed tomography images comprise helical computed 
tomography images • 

58. The method as recited in claim 50 wherein the 
region of interest comprises an air column. 

59. The method as recited in claim 50 comprising a 
data reduction step for reducing the size of the 
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voliime of data before rendering the segmented region 
of interest. 

60. The method as recited in claim 59 wherein the 
reduction step comprises reducing pixel resolution. 

5 61. The method as recited in claim 60 wherein the 
pixel resolution is reduced to 8 bits per pixel. 

62. The method as recited in claim 59 wherein the 
reduction step comprises reducing spatial resolution. 

63. The method as recited in claim 62 wherein the 
10 spatial resolution is reduced to 256 pixels by 

256 pixels. 

64. The method as recited in claim 59 wherein the 
reduction step comprises creating a subvolume of data. 

65. The method as recited in claim 64 wherein the 
15 subvolume comprises the mediastinum. 

66. The method as recited in claim 59 wherein the 
size of the data is reduced to less than about 

10 megabytes. 

67. The method as recited in claim 50 wherein a 
20 portion of the tracheobronchial airway is rendered 

transparent . 

68. The method as recited in claim 50 wherein a 
portion of the tracheobronchial airway is rendered 
semi-transparent . 
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69. A system for interactively displaying three- 
dimensional structures comprising: 

a. volume formation means for forming a 
three-dimensional volume of data from a 

5 series of two-dimensional images 

representing at least one physical 
property associated with a three- 
dimensional body; 

b. segmentation means for segmenting a 
10 region of interest from the volume of 

data based on selected values of the 
physical property representing the 
selected region of interest; and. 

c. rendering means for rendering the 
15 segmented region of interest in an 

interactive three-dimensional display. 



70. The system as recited in claim 69 wherein the 
series of two-dimensional images comprises images 
taken at regularly spaced grid locations within the 

2 0 body . 

71. The system as recited in claim 70 wherein the 
spacing between successive grid locations is selected 
to produce isocubic voxels for three-dimensional 
display . 

25 72. The system as recited in claim 70 wherein the 
series of two-dimensional images comprises images 
taken at 1 mm regularly spaced grid locations within 
the body. 

73. The system as recited in claim 70 wherein the 
30 series of two-dimensional images comprises computed 
tomography images • 
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74. The system as recited in claim 73 wherein the 
computed tomography images comprise helical computed 
tomography images. 

75. The system as recited in claim 69 further 

5 comprising reduction means for reducing the size of 
the volume of data. 

76. The system as recited in claim 75 wherein the 
reduction means comprises means for reducing pixel 
resolution. 

10 77. The system as recited in claim 75 wherein the 
reduction means comprises means for reducing the 
spatial resolution. 

78. The system as recited in claim 75 wherein the 
reduction means comprises means for creating a 

15 subvolume of data. 

79. The system as recited in claim 69 wherein the 
segmentation means comprises a thresholding means for 
determining a threshold range used to isolate the 
region of interest. 

20 80. The system as recited in claim 79 including means 
for producing a wireframe model of the isolated region 
of interest. 

81. The system as recited in claim 79 wherein the 
thresholding means comprises: 
25 a. a threshold selection means for 

selecting the threshold values; and 
b* a threshold adjustment means for 
adjusting the threshold values. 
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82. The system as recited in claim 81 wherein the 
threshold adjustment means comprises: 



5 



a. orthoslice means for taking an 

orthoslice through the region of 
interest to produce an orthoslice 
image; and 



10 



b. display means for displaying the 

orthoslice image and a corresponding 
thresholded image, so that the 
threshold values can be adjusted while 
comparing the thresholded image to the 
corresponding orthoslice image. 



83. The method as recited in claim 22 wherein the 
step of segmenting the selected region of interest 

15 comprises a morphological dilation step. 

84. The method as recited in claim 83 including 
forming an isosurface wireframe model of the isolated 
selected region of interest. 

85. The method as recited in claim 84 wherein the 
20 step of rendering an image includes the step of 

rendering a three-dimensional display of the wireframe 
model. 

86. The method recited in claim 26 wherein the 
thresholding step comprises: 

25 a. a threshold selection step for 



30 



selecting a threshold range 
corresponding to the selected values of 
the physical property representing the 
selected region of interest; and 
a threshold adjustment step for 
adjusting the threshold values to 
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a. orthoslice means for taking an 
orthoslice through the region of 
interest to produce an orthoslice 
image ; and 

b. display means for displaying the 
orthoslice image and a corresponding 
thresholded image, so that the 
threshold values can be adjusted while 
comparing the thresholded image to the 
corresponding orthoslice. 

92. The method as recited in claim 24 wherein the 
step of rendering a three**dimensional display of the 
wireframe model comprises a volume rendering step. 

93. The method as recited in claim 92 wherein the 
15 volume rendering step comprises an opacity adjusting 

step for adjusting opacity values. 

94. The method as recited in claim 93 wherein the 
opacity adjustment step comprises the step of 
adjusting the opacity values to the values determined 

20 along the inverse of a histogram curve representing 
the number of voxels of a given voxel value as a 
function of voxel value. 
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